On sampled metrics for item recommendation

Recommender systems personalize content by recommending items to users. Item recommendation algorithms are evaluated metrics that compare the positions of truly relevant among recommended items. To speed up computation metrics, recent work often uses sampled where only a smaller set random and ranked. This paper investigates such in more detail shows they inconsistent with their exact counterpart, sense do not persist relative statements, for example, recommender A is better than B , even expectation. Moreover, sample size, less difference there between very small all collapse AUC metric. We show it possible improve quality applying correction, obtained minimizing different criteria. conclude an empirical evaluation naive corrected variants. summarize, our suggests sampling should be avoided metric calculation, however if experimental study needs sample, proposed corrections can estimate.